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ABSTRACT 

This is one of a series of reports based on an 
ongoing reality test of systemic evaluation for instructional 
decision making. This feasibility study is being carried out by the 
Center for the Study of Evaluation with the Laboratory in School and 
Community Relations at a suburban Los Angeles high school (called 
Site A). Viewing a school as a cultural/ecological system, systemic 
evaluation is a set of principles to guide local development of 
processes to gather, organize, and utilize information relevant to 
the needs and values of the system and its members. This paper 
discusses four key social contextual issues that emerged during the 
early phases of the Site A project: (1) the current status of the 
school's Computerized Accountability for Student Achievement (CASA) 
information management system for counseling and administrative 
staff; (2) teachers 9 clinical perspective on information and features 
of the school's social organization; (3) faculty-district relations 
as they impinge on the systemic evaluation development! and (4) the 
nature of the school district's support for the project and what it 
portends for the future. (BS) 
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Contextual Influences 1n 
Developing a School -Based Comprehensive Information System 

Donald W. Dorr-Bremme 
INTRODUCTION 

The purpose of this paper is to identify and examine some social 
contextual factors that impinge upon a school's development of 
on-going, systemic evaluation for instructional decision making. The 
data for this analysis come from the case study of a "reality test" of 
such a system now under way in a suburban, California high school. 

The paper begins with a brief description of the school district 
and school in which this effort is being conducted, a succinct review 
of the model of systemic evaluation being tested there, and an outline 
of the reality test itself. It then describes and documents several 
social phenomena that contextual 1ze this effort and goes on to suggest 
how each appears to influence the school's response to systemic 
evaluation. 

All this should be construed as a progress report, rather than a 
list of findings. The reality test (or feasibility study) has been in 
progress one year and has another year yet to run. Nevertheless, both 
experience and related research suggest that the issues identified 
thus far in the project are important ones and that they tend to arise 
in many school districts which are engaged in efforts to develop 
instructional information systems. 

The Setting 

The reality test of systemic evaluation is being conducted at 
Site A in the Valley Unified District* on the fringe of the Los 



* A pseudonym 
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Angeles metropolitan area. The District serves an enrollment of about 
25,000 students, who come from a socioeconomlcally diverse community* 

Site A is one of two high schools in the Di strict ♦ Among Its 
2,000 or so students, approximately 85 percent are "Anglo 11 ; the rest 
are Hispanic, Asian, and Black* School authorities estimate that 8 
to 10 percent of Site A's graduating seniors go on to attend four-year 
colleges or universities, another 20 to 30 percent attend a local 
community college. 

The school prides itself on offering a full range of advanced 
placement courses, a rich curriculum, successful athletic programs, 
and a wide range of other extracurricular activities. Educators at 
Site A and in the District, however, express some concern that Site A 
graduates may not be pursuing post-secondary educational and career 
opportunities concomitant with the quality of school programs. 

While in most ways neither Site A or the Valley Districtare 
unusual, some special resources and programs make them propitious 
settings for a trial of the systemic evaluation concept. 

One distinctive feature of the resources at Site A is the 
presence of a computerized individual information system. CASA 
(Computerized Accountability for Student Achievement) is a 
computer-based student information management system which provides 
counseling staff and assistant principals instantaneous access to a 
wealth of information stored on each student. Through this system, 
counseling staff and adminstrators can access such student-level 
information as standardized test scores for multiple year (including 
information prior to entry to high school); proficiency testing 
information (from the Districts Computer Managed Instruction program, 
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which monitors student progress in Grades K-10 through the use of 
CRT 1 s based on District continuum); curriculum and performance 
information including courses taken, credits, grades and class rank; 
background information including parental occupations, family size, 
census tract location of residence, and ethnicity; current school 
status information including eligibility for special programs (Gifted 
and Talented, Bilingual, Special Education) special school activities 
(athletics, school paper, etc.), complete attendance history and 
referrals to various school services (psychologist, health office 
counselor and guidance office, principal, etc.). 

The CASA system was developed with Title IV-C funds. The system 
is currently being augmented through state school improvement funds 
targeted to the development of a computer-managed Career Magnet School 
(CMS) program. CMS seeks to align student career interests with 
competencies with specific courses at Site A in which they may be 
obtained. Over the past year, teams of teachers have been developing 
the list of skills associated with given courses in the school and an 
analogous list required for each career program. (Careers Magnet 
schools are first broken down into career paths such as computer 
technology and then further into programs such as computer technician, 
operator, programmer, or designer). The skill -to-course and 
skill-to-career matches are being computerized so that counseling 
staff can develop courses of study for students choosing particular 
careers and monitor their progress at obtaining prerequisite skills. 

CMS is one of two efforts recently instituted at Site A that are 
directed at career and academic decision making. The other, the 
Learning Resources Center (LRC), offers students and teachers academic 
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resources (materials, assistance) for remedial and advanced work* 
These two programs appear to be guided by a general concern that 
students do not have the necessary information and skills to attain 
post-secondary education and careers they might warit. The implicit 
assumption is that by providing students with more information about 
career opportunities and their necessary prerequisites (CMS) on the 
one hand, and resources for remediating or enhancing their academic 
performance (LRC), on the other, that students will make better 
decisions about how to benefit from their high school experience and 
be better prepared for their future. 

Thus, Site A in the Valley Unified School District offers a 
setting with a wealth of relevant information already on file and a 
sophisticated system already in place to make that data accessible to 
at least some Site A personnel* It 1s a setting 1n which a few key 
figures had, before the advent of the current project, a keen Interest 
in information for decision making. With these advantages, project 
staff have reasoned, if systemic evaluation cannot become viable in a 
setting with these advantages, it probably cannot become viable 
anywhere. 

Aside from Its promising technology, however, Site A and its 
district provide a typical suburban setting. There is reason to 
believe, therefore, that the kinds of issues, concerns, and 
enthusiasms that surface during the reality test here will be generic 
ones — germane to circumstances in many other schools and districts. 
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The Systemic Evaluation Model 

The model or conceptualization of "systemic evaluation" that is 
in trial at Site A has been outlined by Sirotnik, Burstein, and Thomas 
(1983). Among its key assumptions or features are the following: 

1. Outcome indices have limited value, beyond their immediate 
descriptive signal, for helping direct an agenda for school 
improvement. 

2. A necessary requisite is relevant information on the 
circumstances, activities and sentiments associated with 
schooling process. 

3. The criteria of relevance are based upon the perceived needs of 
the significant "actors" in the setting (e.g., administrators, 
teachers, students, parents) and the inherent value systems 
through which these perceptions are filtered. 

4. Information gathering as knowledge production has several crucial 
and interrelated features: 

a. It is operational!' zed with a multi -method approach to data 
collection (e.g., survey questionaire, interview, anecdotal 
and structured observation, document and archival records). 

b. It is conceptualized and analyzed from a multi -level (e.g., 
individual class, school, district) perspective. 

c. It embraces multi-inquiry paradigms (e.g., empirical 
analytic, natural i stic/i nterpreti ve and critical-dialectic). 

5. Information as knowledge is not an end in itself but is, instead, 
a catalyst for evaluative discourse and action; systemic 
evaluation must, therefore, be legitimized as a natural and on- 
going part of the daily work life of those for whom the knowledge 
is to be relevant. 

Underlying these principles is a view of the school as a cultural/ 

ecological system. Renewal of that system comes about ideally, the 

model suggests, through: 

a process by which the circumstances, activities, 
and meanings [of the school as a system] come to 
be understood and acted upon by people to whonfTt 
is relevant... Lthe renewi process] is people 
actively and continuously engaged in the systemic 
and rigorous deliberation over any and all 
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information seen to be potentially relevant to 
school improvement (Sirotnik, Burstein, & Thomas, 
1983, p. 35) ♦ 

As all the foregoing should indicate, the model or 

conceptualization is not a blueprint or recipe for what to do, but a 

set of principles that can guide local developmental efforts. Thus, 

the "reality test," or feasibility study, was undertaken to learn what 

happens when educators in a school environment actually attempt to 

follow those principles toward development of sytemic evaluation. 

The Reality Test to Date 

The systemic evaluation field test is being carried out by the 
Center for the Study of Evaluation in collaboration with the 
Laboratory in School and Community Relations, both of UCLA. 

While examining the viability of the systemic* evaluation concept 
is the broad goal of reality testing at Site A, there are more 
specific objectives as well. In particular, the project strives to 
explore and work toward resolving methodological issues that arise. 
In view of the nature of the model, two types of methodological issues 
are of central interest. One type can be called technical; the other, 
social . 

Technical issues include problems, concerns, and solution 
strategies that arise in developing methods to generate mul ti -method, 
multi-level data; to integrate that data technologically (e.g., in 
computer files) such that it can address locally relevant concerns; 
and to display that data in ways that different user groups find 
understandable and utilizable. (A paper that accompanies this one 
addresses some of the technical issues that have emerged so far). 

8 



Dorr-Bremme 7 

Social issues include the problems, concerns, and solution 
strategies that surface in developing social organizational 
arrangements — arrangements for identifying "significant actors" in 
the setting; structures through which these actors can articulate 
their information needs and viewpoints on the formatting of 
information; forums and procedures that facilitate consultation and 
use of the information; and arrangements for maintaining the entire 
evaluation-and-renewal system* 

By identifying and beginning to work through these issues during 
the field test, project staff hope to gain information that can help 
other schools and school districts develop their own versions of 
systemic evaluation more smoothly and efficiently. 

To date, CSE project staff have worked closely with school and 
district personnel to carry through the initial stages of this reality 
test* A working group including the Site A principal, other school 
administrators, and a five teachers has met regularly with project 
staff. During the course of these meetings, Site A personnel have 
begun to articulate their information interests and needs, to shape 
and revise a survey of student perceptions and attitudes, and to 
specify ways in which the student survey data and other information 
already in district files can be integrated and displayed for maximum 
utility. Throe reporting formats have evolved. A "Student-At- 
A-Glance" form (including survey data, grade point average, attendance 
and test-score data on each learner) has been designed to accompany 
the class list each teacher receives at the beginning of a semester. 
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A "Class-At-A-Glance" form for each class provides graphic summaries 
of each class 1 perferences among different types of learning 
activities, as well as their overall feelings about the class 
subject. Finally, a "School -At-A-Gl ance" form highlights in graphs 
and prose a set of issues that the working group deems worthy of 
consideration by trie entire faculty and staff. 

At the district level, project staff have examined the current 
district information files and have begun to develop (with district 
data-processing staff) ways of integrating the data in those files 
with the new survey data* Negotiations among project staff, district 
administrators, data processing personnel, and the school principal 
have been aimed at securing continued district support for the reality 
test effort and for its maintenance when the field test ends. 
District officials have reaffirmed commitment to the project, and they 
have become increasingly interested in its data-integration and 
data-reporting dimensions. 

At present, the project has several immediate goals. One is to 
refine the "At-A-Glance" forms and present them to all school faculty 
with actual data for their classes* A second goal is to present 
issues summarized in the "School -At -A-Glance" report to the entire 
faculty and staff in a forum which allows for their initial 
discussion. A third goal is to complete revisions of a teacher survey 
form, administer it to the whole school faculty, and integrate those 
data with information from the student survey and extant district 
information in ways that school staff believe will be helpful. These 
goals wi"l entail continued cooperation with the District office, 
especially the District's data processing group. 

ERIC 10 
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And as these steps unfold, project staff will gather new insights Into 
the technical and social methods that can facilitate the realization 
of systemic evaluation. 

Throughout the project, one staff member has been documenting the 
course of event — contextual features, trends in thinking, 
meeting-by -meeting discussions, etc. — in a nearly verbatim way. The 
600 pages or so of field notes collected this far have allowed UCLA 
staff to reflect on the direction of events over time. Ultimately, 
this case study will provide a record of what has been learned. 

The project also gains perspective on events in the field-test 
setting through recurrent meetings with another CSE effort (the 
Management of Instructional Information Systems, or MIIS, project), 
which has been studying and continues to investigate information 
systems in other school districts. 

This brief sketch of the reality test to date provides background 
for the description and discussion of some key contextual issues that 
have emerged so far. As that discussion proceeds, there will be 
references to and elaboration upon this outline of past and present 
activities. 

ISSUES AND OBSERVATIONS 

Four general features of the district and school context are 
described below, together with their implications for the development 
of a comprehensive information system at Site A. These are (1) the 
current status of the extant district-school information system; 

u 
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(2) Site A teachers 1 clinical perspective on information and the 
features of school social organization that support that perspective; 

(3) faculty-District relations as they impinge on the development of 
systemic evaluation; and (4) the nature of District support for the 
Site A - UCLA project. While these topics are treated separately in 
the pages below, it will soon become clear that the phenomena 
discussed under each heading are quite interdependent. 

The Current Status of the Valley-Site A Information System 

As noted above, Site A has and is continuing to develop a 
computerized student-information management system. Referred to as 
CASA (Computerized Accountability for Student Achievement), this 
system was in place before the systemic evaluation reality test 
began. One might imagine, then, that CASA served to break ground for 
the development of systemic evaluation at Site A: that it 
familiarized Site A staff members with the advantages of easily 
accessible information for instructional planning and decision making; 
that it generated enthusiasm for and sophistication about information 
systems and their use; or that it led to technological and 
organizational procedures, upon which systemic evaluation could build, 
for accessing and using information. In fact, however, the presence 
of CASA has had little discernible impact to date on Site A's progress 
toward comprehensive, systemic evaluation. A brief review of CASA's 
history and current status reveals why; it also underscores some of 
the conditions necessary for instructional Information systems to be 
useful and utilized. 
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Like many educational Information systems (Bank 4 Williams, 
1982), CASA emerged from the Interest and enthusiasm of a very small 
group. One Site A assistant principal championed the Idea of 
accessing the District's huge data base for Instructional planning at 
the school level. She took the initiative 1n CASA's creation. Later, 
as the Career Magnet School (CMS) concept evolved, this same 
administrator saw CASA's potential for facilitating it. Together with 
the Site A improvement program (SIP) coordinator, she wrote the plans 
t„dt secured state SIP funds for Unking CASA with CMS. (Refer to 
page 4 above for details.) A few others 1n the high school's central 
office, counselors and administrators, offered their support and 
suggestions as the present CASA system was developed. Evidence 
suggests that no faculty members were involved. 

Thus, CASA was 1n no sense a response to a schoolwide interest in 
Instructional information systems, 1n no sense a product of schoolwide 
demand for better or more easily accessible information about 
students. Among Site A's faculty and staff in general, no such 
interest or demand was expressed. Furthermore, with one exception, 
the few central office personnel who promoted and developed CASA did 
not seem to think of it as a schoolwide Innovation. Certainly, there 
was no schoolwide planning to Inform its design. 

As it was originally intended to be, CASA has been used primarily 
by counselors and assistant principals. They draw upon it as they 
advise students on academic and career paths and as they handle 
referrals of students with Individual problems. 
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CASA is also available to classroom teachers. Any member of the 
faculty can go at any time to the central office, where one of the 
school's administrative assistants has been designated to process 
their CASA requests. Through her, they can obtain' information on 
individual students or class groups in any of a wide variety of 
standard report formats. (For a list of some of the data available 
through CASA, see page 4 above.) Nevertheless, all the evidence 
available reveals that very few teachers have made use of this 
opportunity. The following excerpts from the project's case study 
field notes document, and suggest some of the reasons for this 
phenomenon. 



NOTE #1 (February 22, 1984) . The first working-group meeting of the 
new project year is under way. Five teachers, an assistant principal, 
the school improvement coordinator, and the administrative assistant 
who coordinates CASA requests are present, together with the UCLA 
staff. Discussion has been following the first three items on the 
agenda prepared for the session by UCLA: "(1) Identify the kinds of 
information teachers, counselors, and building administrators view to 
be useful for their work... (2) Identify what specific problems (at 
any level) Site A staff would expect the information system to help 
them address; (3) Ascertain level of understanding of the computerized 
information currently available to the school staff..." The teachers 
present have mentioned many kinds of information they feel would be 
beneficial to them: whether students in their classes have course 
prerequisites, students 1 past grades in their departments 1 courses, 
students 1 reading level, students 1 writing ability, and students 1 
vocational goals. At this point, UCLA project co-director Leigh 
Burstein observes that nearly all this information is currently 
available to teachers on one of the standard CASA report forms known 
by the acronym ARF (Activities Referral Form). A foreign language 
teacher in the group responds, "I didn't know we could get that." 
Other Site A faculty in the group murmur their assent. Leigh Burstein 
describes the Activities Referral Form in more detail. The foreign 
language teacher asks, "Now why haven't we heard about that. That 
sounds very good." One or two other Site A staff members agree. A 
member of Site A's English Department replies, "I think we did hear 
something about that. Wasn't there — I think there was, uh, a 
presentation in a faculty meeting last spring about it." Someone else 
from Site A adds, "Oh, I remember that!" The Math Department 
chairperson comments with a chuckle, "It certainly made a strong 
impression, didn't it?" 
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NOTE #2 (February 29, 1984) . The second working-group meeting is in 
progress with the same group members as in the first session a week 
earlier. The principal, however, is in attendance this time. Site A 
staff members in the working group have produced lists "for items to 
be printed on one sheet of information, to be distributed to teachers 
one time per semester (one sheet per class)," as one group member has 
labeled her list. The suggestions on each list, which represent group 
members 1 own ideas and those of colleagues with whom they have spoken 
during the past week, are now under discussion. As the foreign 
language teacher finishes presenting her suggestions, the following 
exchange ensues: 

Health Teacher : "It seems to me, urn, isn't this just 

computerizing the cum. folder and information card?" 

Math Department Chairperson : "Well, it seems to me part of the 

problem Is that all tne information people want isn't in one 

place. It's in several locations. This would bring it 

together." 

Health Teacher : "Well I don't know that. I don't know where all this 
information is. I don't even know what questions to ask to get the 
information I want." 

Math Dept. Chair : "You don't know whether to ask for an ARF or a 
RALPH or. .. (general laughter). I'm not sure I do, either. 
Health Teacher : "That's it. That's it exactly." 
Later on in the meeting, discussion again returns to the ARF, or 
Activities Referral Form. UCLA staff have distributed an example of 
one ARF to the group. 

Leigh Burstein (UCLA) : "So far ARF is holding up pretty well, this 
information, the information we : ve been talking about, 1s in there. 
But the display issue will show up, because you want lots of 
information and it's not going to be easy to get it all clearly on one 
page." 

Ken Sirotnik (UCLA) : "This format [holding up the ARF] is too dense 

already. That may be why it isn't used." 

Assistant Principal : "Well, it's quite new, too." 

English Teacher : "")lo, he's right [holding up her copy of the ARF.] 

And it has to be in Engl ish . With all these numbers and codes, what 

have you — it won't work like this." 

Several others from Site A's staff nod and vocalize their assent. 
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NOTE #3 (May 23, 1984) , A student survey developed by the systemic 
evaluation working group is about to be administered in classrooms 
throughout Site A. To prepare the way for this, UCLA staff are 
spending the day meeting with small groups of Site A faculty. During 
each period of the school day, those teachers with no assigned class 
meet with UCLA staff in a corner of the Learning Resource Center. 
There, Ken Sirotnik explains the history and purposes of the sytemic 
evaluation effort in general and the student survey in particular. He 
also answers teachers' questions, listens to their reactions, and 
outlines procedures for administering the student survey. Nearly 
every teacher in the school, plus some of the counselors and assistant 
principals, makes an appearance in one of the six sessions. In all 
but one of the six, there is considerable discussion and the exchanges 
are (in the jargon of international diplomacy) "frank." 
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Throughout the entire day, despite the fact that many of 
Sirotnik' s remarks in each session could easily "cue 11 teachers 1 to 
think about CASA, only one teacher makes any reference to the CASA 
system, ARF, or any currently available information on students. 
Ken Sirotnik (during Period 4, finishing his initial explanation of 
the project and the rationale behind the student survey): "So, we 
will report the information back to you, and we may do so in several 
TTFFerent ways that you can use, depending on what your colleagues who 
are working with us suggest. But I guess before I go on, now would be 
a good time to talk about issues ^ou want to raise, and to hear your 
comments. 11 

Teacher : "You should know that we don't have a good record on getting 
information back. I was looking for correlations for my class on that 
[CASA] request form from [the administrative assistant who handles 
CASA requests]. I haven't gotten it yet. I think it's been over a 
week." 

Sirotnik : "Well, there may have been problems there, and that's one 
o1^ the things we're working to avoid..." 

Teacher : "OK. I'm just pointing out that what we have is it's not 
accessible now." 

These incidents strongly suggest that the CASA information system 
is neither well known now widely used among faculty *t Site A. They 
give life and support to a comment dropped casually during the course 
of one meeting (with District personnel, on February 8, 1984) by the 
Site A administrative assistant who handles CASA requests. "We 
haven't been getting a lot of requests from teachers," she said. 
Indeed, the notes above suggest that this remark is something of an 
understatement. 

Faculty members in the Site A-UCLA working group ~ people who 
demonstrably have a special interest in information that can help them 
in their work, who care enough about obtaining such information to 
give their time to the project, and who volunteer that they would like 
to have some of the information now available in the CASA system — 
could not recall even hearing about CASA's availability. In the 
course of meetings with small groups of teachers, meetings in which 
each time teachers heard a description of a project very similar to 
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CASA in some of its goals and paticular features, no one asked, "Don't 
we have something very similar to this?" or "How is what you're 
talking about different than what we can get now?" Surely they would 
have done so, amidst the frank discussion that occurred, if they were 
aware of CASA. Given many teachers 1 expressed concern with the time 
costs and ultimate value of "one more survey," this seems especially 
likely.* Finally, we have the administrative assistant's report 
(confirmed by District data processing personnel) that few teachers 
have called for CASA reports. All of this indicates why the presence 
of CASA has done little to pave the way for the development of 
systemic evaluation at Site A. CASA has, by and large, remained 
extrinsic to the professional lives of nearly all of Site A's 
faculty. 

It is not difficult to account for this state of affairs. Pieces 

v/i a it c/\p i uitout vit uic v-Hi cuucu uitt uu^n uuc; u t i u i it i o wi j uuu 

field-note excerpts provided above. 

First, and probably most importantly, nothing was done at Site A 
to build teachers 1 investment in CASA or to assure that CASA 
information and report formats responded to teachers 1 needs. There 
were, as we have seen, "good reasons" for this. CASA was not 
construed as a schoolwide organizational innovation. It was developed 
as a convenience, a technological short cut to facilitate central 
office and counseling functions. Teachers were told about it after 
the fact, rather than involved in its development. (Furthermore, 
within the flow of information that teachers constantly encounter in 
faculty meetings, this "telling" was evidentally not an especially 
salient event.) Thus, all the recent work on innovation and 
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dissemination in educational environments work captured in such 
buzz words as "stakeholders n ownership ," and "mutual adaptation" 
helps to explain why CASA has not "caught on" throughout Site A. 

Second, there is evidence that the formal features of CASA (as 
opposed to its content) do not strike teachers as immediately . 
appealing. Working-group members, viewing the ARF report apparently 
for the first time, found it unattractive and hard to read (Note #2 
above). At least one teacher who requested CASA data found that he 
did not receive it promptly. Such shortcomings as these can be 
traced, at least in part, to the absence of faculty participation in 
CASA's development. They may also be factors in the limited teacher 
use of CASA. 

A third factor, not addressed in this account so far, may also 
have contributed to CASA's general confinement to the central office 
and to counseling tasks in particular. Teachers at Site A seem to 
take for granted that the functions of counselors and the functions of 
classroom teachers are generally quite separate. Thus, some teachers 
seem to feel that teachers need no more information about students to 
perform their roles. As one put it after hearing Sirotnik's expla- 
nation of the project in the small group meetings (Note #3 above): 

I don't claim to understand all this but it seems 
idealistic. This is going to be a great tool for 
counselors, but I don't see how I can use 1t as a 
teacher. 

What 1s more, teachers on more than one occasion have expressed 
frustration with the operations of the counseling office. Near the 
very outset of the working-group meeting described above in Note #1, 
for instance, as teachers were voicing some of their information needs 
off-the-cuff, one teacher said: 

ERIC 18 
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Let's get specific here. There's some things 
going on in the counseling office that are just 
very hard to understand. For instance, I teach 
social psychology, which is supposed to be a 
required course. But there is a procedure to 
waive the requirement. I'm interested in whether 
the kids who 1 re having It waived aren't the ones 
who really need that course. How can I get that 
information? I assume they have it in counseling, 
but how can I get it? Then, I get kids who 1 re 
actually jni the class who're on the waiver list... 

This remark set off a chain of anecdotes from other faculty in the 

working group about anomalies in course assignments and counseling 

decisions. The episode only concluded when the original speaker 

quoted above suggested that an information system should "kick out 

anything unusual, like the IRS computer does with tax returns, because 

obviously they're not doing this down there [in the counseling 

office]." To this, there was general agreement in the group. 

These and similar events indicate that in Site A's informal 

social structure, a boundary that is only semi -permeable niarks off 

teachers and their concerns from counselors and theirs. This 

boundary, it appears, is reinforced by some strong teacher feelings 

about the quality of counselors' work. In the experience of this 

author, such a boundary is hardly unique to Site A; similar divisions 

exist in many American high schools. Nevertheless, its apparent 

presence at Site A may well have truncated the teacher- counselor 

communication through which information about CASA and its benefits 

might otherwise have been shared. The informal networks for such 

communication seem to be absent, and many teachers appear to believe 

that what is good .for counselors is not necessarily advantageous for 

teachers. 
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This analysis of the history and current status of Site A's 
current information system provides some insights into the social 
milieu that surrounds the development of systemic evaluation at Site 
A. Principally, it should make clear that despite the existence of a 
relatively sophisticated information system at Site A, most Site A 
staff are neither enthusiastic nor sophisticated about instructional 
information or information systems. Furthermore, it should help to 
demonstrate that the social-organizational structures implicit in the 
systemic evaluation model described earlier have not yet been built at 
Site A. 

Viewed in broader perspective, the account in this section begins 
to show that for all practical purposes, there is nothing special 
about the current status of information and information use at Site A. 
Most districts and schools do much less than they could with the 
evaluation and assessment data they have on hand (e.g., Bank & 
Williams, 1981; Dorr-Bremme, Herman & Doherty, 1983). Those that 
routinely and systematically link it with instructional planning or 
school improvement are the exceptions. 

The next section of this paper explores how teachers at Site A 
think about information and their information needs under these, very 
usual, curcumstances. It also suggests how some of Site A's 
organizational arrangements support these ways of thinking. 

The Clinical Perspective and School Social Organization 

The concept of systemic evaluation that is generally guiding the 
Site A reality test (as outlined above, pages 6 to 7) places 
considerable emphasis on the social uses of information. 
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Information use by individual teachers and others is certainly 
included as part of the evaluation and renewal process. Nevertheless, 
information is viewed expecially as a "catalyst for evaluative 
discourse and action" (Sirotnik, Burstein, 4 Thomas, 1983, p. 4), 
The renewal process is construed as "systematic and rigorous 
deliberation over any and all information seen to be potentially 
relevant to school improvement" (p. 35). Such statements as these not 
only call attention to the importance of the social uses of 
information in the systemic evaluation model; they also reflect its 
commitment to a holistic renewal process. The school is viewed here 
as a cultural/ ecological system. Its parts are conceived to be 
interdependent. Thus, it is ideally the entire system upon which 
renewal efforts focus. This means that information use toward renewal 
is necessarily a social process, in which all relevant actors engage. 
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Especially in light of these ideals, it is important to note that 
Site A teachers do not routinely or easily think of information in 
terms of its social uses. Rather, they tend to approach information 
and their information needs from a clinical perspective. 

Two hallmarks of the clinical perspective, according to 

sociologists of applied knowledge, are its orientation toward action 

and its emphasis upon the individual case. Elaborating on these 

points, Homans (1950) explained: 

Clinical science is what a doctor uses at his 
patient's bedside. There, the doctor cannot 
afford to leave out of account anything in the 
patient's condition that he can see or test. It 
may be the clue to the complex... In action we 
must always be clinical. Analytic science is for 
¥ n understanding but not for action. 
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Noting with Homans that the aim of the clinical practitioner "is not 
knowledge but action," Friedson (1970) adds that n the clinician is 
prone in time to trust his own personal first-hand experience" and to 
be "particularistic," stressing the uniqueness of each case to be 
treated. The "clinical rationality," Friedson (1970, p. 171)- 
concludes, "is particularized and technical: it is a method of sorting 
the enormous mass of concrete data confronting [the practitioner] in 
individual cases." 

it is the clinical orientation as defined here that character- - 
izes the thinking of Site A teachers participating in the systemic 
evaluation reality test. Their central interest is particularistic. 
They want to know primarily about "this student" or (secondarily) 
"this class"; rarely do they manifest spontaneous interest in knowing 
about the students in "this school," or even those in a given 
department or program. They require that information be relevant to 
action, recurrently asking "What can I do with that?" and declining to 
gather information because "I can't do anything with it." In 
particular, they seek information that supplementSj and helps them 
sort and clarify, the plethora of personal, first-hand information 
they gain about particular students and class groups as they interact 
with them. 

The project to date has gathered a wealth of data to substantiate 
these generalizations. Only a small portion of it will be reviewed 
here. 

NOTE #4 (Fefarary 22, 1984) . Leigh Burstein (UCLA) has opened this 
first meeting to discuss "what information you'd like to have 
available" by underscoring the many levels at which data can be 
aggregated to address needs of different types. He mentions 
"information for or on specific programs" and calls attention to the 
importance of data for "constant monitoring at the school level — 
course enrollments, drop out rates." He expands on his belief that 
information at the school level can help in the "planning and design, 
in _studying_the_ impact of new programs you want to start. . . In my view 
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this is what an information system should do." Two hours pass and the 
group takes a break. Up to now none of the teachers ment oned 
anything °*™ P t information they'd lik e to nave on their individual 
students an d classes , the assistant principal, In a brief cornnent, 
h^exoressed interes t in "a graph that makes attendence very visible, 
5hat SJld show the match with time of day, day of week neighborhood 
grid. This would help in working on attenaence with neighborhood 
organizations and elementary schools. 

The group returns from the break and continues to discuss 
information on individual students and classes. As the 3:30 p.m. time 
for drawing the meeting to a close arrives, Burstein again encourages 
the Site A participants to consider "measuring school climate issues 
and the "on-going monitoring" functions of a comprehensive evaluation 
system. He assigns each working-group member to come into the next 
meetinq with a list of questions or issues (1) the kinds of things 
[information] you need for your students, classes, departments and so 
on or; (2) the kinds of things you'd want to collect at the school 
level on an on-going basis." The meeting ends with no further 
discussion of school -level information. 

Despite considerable prompting to consider other levels of data, 
then, teachers in this initial meeting framed their discussion of 
information needs exclusively in clinical terms. Their interest were 
in data on individual students and particular classes. And despite 
Burstein's assignment in preparation for the next meeting a week 
later, that meeting too focused almost exclusively on clinical 
information needs. 

NOTE #5 (February 29 1984) . The meeting opens with a discussion of 
the foreign language teacher's list of desired Information. ,Refer to 
Note #2 for context.) The list focusses on individual student 
characteristics. Ken STrotnik (UCLA) then asks for other ideas. 
Social Studi es Teacher : "I was talking to the people in my 
department, and tne most important thing people want to be able to do 
is to see what their class is like, so they can group kids for 
cooperative learning, so they don't assign work that s beyond the 
kids' level, things like that." 

Ken Sirotnik : "There's not some other information that you, as a 
teacher, tmnk is more valuable...?" 

Social Studies Teacher : "Sure, there's lots of information that s 
valuable. And sure, 1 can wait for the class to begin and see the 
kids work and then I get that information. But what they want, what 
we want is for the beginning of the semester, when you don t know your 

SiJotnik: "Would they want students' GPA [grade point average] in the 

general area?" , , ^ . . 

Social Studies Teacher : "No, people didn t want to get that 
specific. What you need to know is what level are kids reading at? 
What are their comprehension skills? How well can they write? 
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English Teacher : "Yeah, just a rough picture* You can modify it once 
you start woriang with them." 

Assistant Principal : "You wouldn't want their 6PA in the general 
area, as Yen was suggesting?" 

Social Stu dies Teacher : "No, but I'd like to have their grades in 
particular KngHsh classes. That alerts me to the kinds of success 
they have. It tells me not only how well they're doing but the Kinds 
of strengths and weaknesses." ' 

A few moments later, the foreign language and social studies 
teachers turn to explaining some of the value of background data on 
idividual students. , _ 

Soc ial Studies Teacher : "What we need is information that lets us 
respond to the kid who says, when you give them the assignment, I 
can't read five pages.' you want to be able to go to your list [a 
single page with information on each student in the class] and say, 
'That's not what this tells me. You're reading scores show you can do 

thi s . . . ' " . _ . . . 

Foreign Lan guage Teache r: "Right. 'And it says here [gesturing to 
imaginary information sKeet] you have no job; you're in a college prep 
program, so I don't see a problem.'" 

Still further on in the meeting, UCLA staff try to turn 
discussion away from information on individual students and classes 
and toward consideration of school-level data needs. Sirotnik, for 
instance, argues that information on students' preferred learning 
methods and strategies" would need to be "content-free If it s to be 
useful schoolwide." Three teachers immediately respond that (as one 
nut it) "vou can't ignore the subject matter if this is going to help 
us "pi an our classes." Burstein suggests that you could bank questions 
on instructional practices as part of an on-going effort to track the 
health of the school." As an example of such questions, he points to 
a Study Of Schooling survey the group is using as a stimulus for 
iHeas. The agree-disagree questions listed there include such items 
as""The teacher gives me too much work to do in this class"; "Students 
know the goals of this class"; and "The teacher tells us how to 
correct mistakes in our work." Burstein explains that these could be 
asked about "the teachers in this school in general," instead of about 
particular teachers in particular classes, as in the original. Site A 
teachers reply that this Information would bs seen as "too 
threatening" if gathered about individual teachers and that it 
wouldn't be useful to anyone" if students answered about teachers in 
general. Teachers fears are discussed, and the meeting ultimately 
ends with no further discussion of school- or program-level 
information. 

These transactions demonstrate the persistence of teachers' 
clinical thinking. As in Note #4 above, the Site A teachers in the 
working-group and the colleagues with whom they spoke simply could not 
independently generate suggestions for data that would be worthwhile 
at the school level. Here too it is evident that they want 
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information that can help them make sense of and respond to individual 
cases: information for planning their class; information for respond- 
ing to the student who says "I can't read five pages"; and so on. 
Furthermore, they want this particularistic information at the 
beginning of the semester. Once teaching and learning are under way, 
they will have additional information on students from their own 
experiences with them. This information will allow them (as one 
teacher maintains) to "modify" the general view of the individual 
learner that they can obtain from test-score and 6PA data. Thus, the 
clinicians' tendency to trust personal, first-hand knowledge comes 
through, as well, in these conversational exchanges. 

The clinician's action orientation is evident in all the above, 
but it is especially. apparent in the following. 

liOTE #6 {April 3, 1984)= The group is working its way through 
student attitude surveys originally used in the Study Of Schooling 
research, selecting items and issues that seem likely to generate use- 
ful information that can be tied to the District's huge data file. 
About twenty minutes have gone by when the group turns to a set of 
aqree-disagree questions headed, "Relation to Other Students." Among 
the six items are such statements as "I'm popular with kids my own 
age" and "It's hard for me to make friends." Introducing their con- 
sideration, Ken Sirotnik (UCLA) explains that they "cluster to yield a 
score which you could call 'self-concept toward others.'" A debate 
erupts about who would use this information. The foreign language and 
health teachers maintain they're not interested. B i wouldn't have any 
need for that," says the former. Leigh Burstein (UCLA) argues that 
this data could shed light on the school -wide attendence problems 
"you've all been concerned with." He also notes it could be used in 
an on-going monitoring of the health and climate of the school. Ken 
Sirotnik adds, "The question here may be not so much what you're going 
to do with this in your class, but what's a whole faculty going to do 
if they find many students have a low self-concept, there's attendence 
problems at the same time. Shouldn't they know that?" 
Health Teacher : "Look, what I'm saying is who uses this? If I were 
starting a school, I might want know this, but who is there right 

now? " 

Social Studies Teacher : "I think the counselors might want it. Id 
want it if I were a counselor." 

Foreign Language Teacher: Even though I didn't choose these, I have 
no objection to asking them. 
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Math Dept* Cha ir: "You could use this for longitudinal monitoring, 
though, like Leigh has said, couldn't you? 11 
Burstein: "I hope so/ 1 

Health teacher : "That seems like we're just collecting information 
for tne sake of collecting information." 

English Teacher : "No, I think this could really help with the 'Track 
A kids, in confluent education* I'd want to know how students 
feel about themselves ♦" 
Discussion passes on without a consensus. 

Soon attention turns to a set of questions that elicit students 1 
view of the quality of the schools "physical plant." Everyone agrees 
this is within the administration's purview. The issue of whether to 
include these is resolved when one teacher says, "Let [the assistant 
principal] and [the principal] decide if they want it." 

Near the end of the meeting, a set of questions about why 
students elect the classes they do is examined. Several teachers 
point out that since the school administration has just made decision 
to limit students 1 choice of classes, this information is irrelevant. 
"We can't do anything about this," the social studies teacher 
reasons. Burstein again raises the importance of considering the 
value of this information in a longitudinal sense," but the teachers 
end up rejecting the items as useless. 

As noted above, the views expressed in this excerpt highlight the 
action orientation of the clinical perspective, as manifested in the 
concerns of Site A teachers. Together wit*, the other field notes 
excerpted and transcribed here, this helps to document that Site A 
teachers do indeed approach information and their information needs 
from the clinical perspective. 

There is nothing especially surprising in this finding. Teachers 
across the nation seem to adopt a clinical stance in seeking, 
interpreting; and using data about students (e.g., Dorr-Bremme, 
1983). Nevertheless, the account presented here suggests that a 
substantial gap exists between teachers' routine ways of thinking 
about information, on the one hand, and, on the other, the way 
systemic evaluation principles posit that they should. This gap is 
currently an important part of the context in the development of 
systemic evaluation at Site A. What is more, if the national data 
just referenced are accurate, it promises to be a key contextual 
factor in many schools and districts. 
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While Site A teachers perspective on Information 1s a particular 
kind of perspective (a clinical one), it also has certain generic 
features. Like members of other organizations, teachers at Site A 
are interested 1n information that has "theoretical or practical 
Import for organizationally relevant purposes and routines 11 
(Garflnkel, 1967, p. 191). It simply happens that, given the social 
organization of Site A, the only organizationally relevant purposes 
and routines for teachers there are clinical in nature, I.e., taking 
action toward individual students and class groups. At present, the 
school maintains no regular organizational structures that bring staff 
members together and empower them to discuss and resolve common 
concerns. 

As most high schools in the United States are, Site A is 
organized Into various academic departments. Department meetings 
occur, but they apparently do not consitute settings for dialogue and 
conjoint decision making. When Bursteln suggested that some type of 
survey data might help departments plan their curricular emphases, 
faculty 1n the working group rejected the notion Immediately. One 
teacher explained that "everyone sort of sidesteps disagreements over 
teaching methods and philosophy and things" during department 
meetings. Another added that departments meet infrequently and 
usually deal only with what courses individual members of the 
department want to teach, what books they want to order, and similar 
routine tasks. In another working group session, Burstein asked 
whether "your departments" could use information on students 1 
perceptions of Instructional practices. After some initial confusion 
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about what Burstein had in mind (several Site A participants asked in 
apparent disbelief, "Why?" and "For what?!"), one teacher answered, 
"No, this would be seen as threatening." The matter rested there. 

Site A, as noted in passng earlier, participates in the 
California School Improvement Program (SIP). SIP guidelines require 
schools to assemble a school site council which includes the principal 
and elected representatives of various constituencies: teachers, 
other staff (e.g., counselors, non-certified personnel), parents and 
other community members, and students. According to SIP provisions, 
the site council has responsibility for assessing school wide needs, 
developing improvement objectives, planning activities to meet them, 
and evaluating the results of these efforts. SIP Schools are 
encouraged to engage in comprehensive planning and to use the planning 
process as a catalyst to or motivator for dialogue and involvement. 
Some schools do so, but many treat the planning process merely as a 
hurdle that must be jumped in order to procure additional state monies 
(Dorr-Bremme, et al., 1979). Site A seems to fall in the latter 
category. Several discussions of Site A's SIP program during 
working-group sessions suggested that a few administrative leaders 
have primary responsibility for SIP plans. Thus, the SIP site council 
does not appear to provide a forum for substantial teacher involvement 
in schoolwide planning and decision making. Similarly, other Site A 
instruction-related programs (the Career Magnet Schools and Learning 
Resource Center programs, for instance) appear to be the artifacts and 
concern of a few key administrators. 
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More generally, role boundaries are relatively well defined at 
Site A. Teachers teach; counselors advise students, help them plan 
their programs, and deal with special problems; administrators set 
policy and concern themselves with school wide issues. That faculty 
members currently tend to accept and cooperate in sustaining these 
boundaries should be evident in some of the remarks quoted earlier. 
(Teachers referred the decisions on whether to gather certain student 
attitude data to counselors and administrators, for example. See Note 
#6.) 

It is not the case, then, that teachers are simply predisposed to 
see information and their information needs from a clinical 
viewpoint. Their clinical perspective is in fact supported by the 
organizational arrangements within which they operate each day. Those 
arrangements provide no occasion for using information socially; they 
generate no need to consult school-level information. Thus, when 
teachers consider the marginal utility of new information — when they 
implicitly and explicitly address the common organizational question, 
"Will it have been worth the cost to gather this data?" — the 
criteria they employ and the decisions they make reflect the practical 
contingencies and exigencies they face on the job (c.f., Dorr-Bremme, 
1983; Garfinkel, 1967). They can use additional information to 
operate clinically in the classroom. They cannot use information for 
anything else. 

It follows from this analysis that Site A's organizational 
structure is every bit as important a contextual influence in the 
development of systemic evaluation there as teachers' clinical 
perspective is. Indeed, this analysis suggests that the two are 
ERXC interdependent. Other CSE/UCLA research supports this view. In 
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studies of testing-evaluation-instruction linkage systems in school 
districts, Bank and Williams (1983) found that organizational 
supports for information use and teachers' attitudes toward 
information tend to evolve together and be mutually supporting. 

Faculty - District Relatio ns: Conflict and Trust 

The current status of the CASA information system, teachers' 
clinical perspective on information, and the organizational 
arrangements that help sustain the latter are aspects of the school 
context that have already begun to influence the systemic evaluation 
field test at Site A. Now, attention turns to contextual factors 
which promise to influence the field test in the immediate future. 
One of these, discussed in this section, is the demeanor of the 
faculty at large toward the evolving systemic evaluation innovation. 

Recall that to date only a small number of Site A faculty and 
administrative staff members (the working group) have been intimately 
involved with the project. The remainder of the faculty has 
participated only tangentially in two ways. First, they attended 
small group meetings during which the systemic evaluation project's 
history and goals were described. (See Note #3 above.) Second, they 
administered a student survey during one class period. 

Soon, however, the faculty and staff at large will begin to play 
a much more important role in the project. On November 7, they will 
attend a three hour faculty meeting devoted exclusively to the 
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systemic evaluation project. During that meeting, they will review 
and react to the "Student-At-A-Gl ance" and "Class-At-A-Glance" forms 
developed by the Site A - UCLA working group. They will also examine 
four key student-survey findings incorporated on the 
"School -At -A-Gl ance" reporting form and begin to consider whether 
these merit schoolwide action. (These forms are described above on 
pages 8 and 9.) A teacher survey will be administered later in the 
year. And, assuming continued District support, all teachers will 
receive Student-At-A- Glance and CI ass-At-A-Gl ance data for each of 
their classes at the beginning of the second semester in the present 
school year. Faculty reactions to the systemic evaluation project as 
a whole will soon become extremely influential in the course the 
project takes. The responses of teachers throughout the school will 
also, of course, contribute significantly to the learnings that the 
reality test has been designed to engender. 

Under these circumstances, it is especially important to ask, 
"What 1s the present outlook of the faculty? How are they now 
inclined to view activities of the type that systemic evaluation, as 
developed at Site A, is likely to entail for them?" 

With respect to the kinds of information teachers in general are 
most likely to find interesting and relevant, there is every reason to 
believe that viewpoints of faculty members in the working group are 
likely to be broadly representative. This is also true of the formats 
in which they are most likely to find that information appealing and 
useful. As noted earlier, working-group teachers have tried to speak, 
at least in part, for the faculty at large. On at least one occasion, 
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they elicited the views of others in their departments (see Note #5 
above). Thus, the particularistic, clinically oriented information 
presented in the concise, Student-At-A- Glance and CI ass-At-A-Gl ance 
forms seem to be reasonable "first draft" products to offer the 
faculty as a whole for their review. 

At the same time, however, working-group teachers recurrently 
caution that their perspectives are not shared by all faculty 
members. Some, they point out, have no interest at all in survey 
information on students' feelings and viewpoints. As one teacher 
explained during a working group session: 



I'm interested in what students like to do in 
class and what methods work best for them, but I 
don't think all faculty would be interested in 
this. Many of them would say, "when the bell 
rings, I'm going to close that door and teach what 
I want to and the way I want to based on what J_ 
think is best. 

On another occasion, during a working-group discussion on the value of 
gathering data on students' views of teachers' helpfulness, a 
different member of the working group opined, "There aren't probably 
ten people throughout the school at this time who care anything about 
that." 

Even such "hard" data as test scores showing students' reading 
level and writing ability, teachers in the working group warn, is not 
of universal relevance. 



If I were to ask the question, how many teachers 
will use this, I'd have to say, well a small 
number. You've got to remember there are 93 
teachers out there and in some areas — in PE, 
horticulture, and some others -- they don't 
concern themselves with this stuff. 
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In short, working group teachers emphasize, as one might expect, 
that every faculty member will not be interested in a given type of 
information about students and some may not be interested in any 
student information at all. 

But the particular information preferences of Site A teachers are 
less important, at this point in the project's development, than their 
general stance toward the project and its data collection and 
reporting activities. To engage in dialogue with UCLA staff and Site 
A working group members, to be willing to give the project their time, 
Site A faculty and staff must first believe that some kind of data 
collection and reporting activity can be worthwhile. Put another way, 
they must be more-or-less convinced that the benefits of a project 
such as this one can be worth the costs ~ the time and energy they 
will have to invest in it. It appears that at present not all Site 
A's professional staff are. 

During their initial introduction to the project in small-group 
meetings on May 23, 1984, a notable minority of teachers voiced doubts 
that benefits would come from the systemic evaluation effort. Such 
doubts seem reasonable in light of the experiences they report, as the 
following quotations from participants in the May meetings reveal. 

- Ttiere were four surveys already this year. We 
never saw the results of any of them. 

- When [the new principal] was here [for a visit] 
last spring, he gave us a questionnaire. Has 
anyone seen the results of that? 

- You'll find your facing a negative atmosphere 
here. The teachers have been "statistized" to 
death. This [student questionaire] is just one 
more survey. There has been no follow up from the 
district on any of them yet. 
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- We gather a great deal of useless Information here 
alreadly... As long as the teachers 1s powerless to 
act on the Information about students, then 1t all 
just disappears 1n quicksand* It's more 
frustrating than anything else 1f you can't do 
anything with it. 

- I know you [UCLA people] need this for your 
professional careers; It's nothing personal, but 
many teachers have had too much of ed. school 
people who do research and then run off and give 
boring lectures e And who does it help? 

While these remarks explicitly express skepticism about the 

benefits of any data collection enterprise, implicitly they also 

reveal concerns about costs. As yet another Site A teacher explained 

during the May 23 meetings: 



It's not just you, but you're one of 900 separate 
projects with demands... We've had Students 
Against Drunk Driving, substance and drug abuse 
projects, and each one got their time 1n my class. 

Still others who spoke up in the small -group meetings worried 

that the data would be invalid, negative, and bad for the faculty. 

"You're going to get automatic reponses on this," one explained to the 

UCLA speakers. "The majority of kids will respond in very negative 

ways that they may not really feel." Said another, referencing 

changes in school policy and procedures under the new principal, "This 

1s a year of change and it's a bad time of the year, too. It 

guarantees negatives." Yet another argued: 



We have a history of being told the results of 
everything in a way that points out our 
weaknesses. The kids have just been taking tests, 
and regardless of the results, they'll find fault 
with us somehow. What they'll say about our way 
of giving tests will be negative. 
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"If this gets out," warned still another teacher, "it's going to just 
be more fodder for the press to condemn us with." 

Working-group teachers, who participated in the May small-group 
sessions with their colleagues, attempted to put all these negative 
initial reactions in context for the UCLA staff. Faculty morale at 
Site A, they explained, was very low in general. More specifically, 
working group teachers went on, many teachers were afraid that the 
data would be used against them. Recent events in the District were 
at the root of all this, they added. 

It seems that during the 1982-83 school year, the Valley Unified 
School District felt impelled to reduce the number of faculty it 
employed districtwide. Even some tenured teachers had to be let go. 
Teachers found the Districts "riffing" (or reduction in force) 
procedures highly unfair. Within-di strict transfers that accompanied 
the staff reduction, they maintained, resulted in teacher assignments 
"that make no logical sense." 

Part of the "riffing" process involved the use of information. 
According to one teacher, the 153 district faculty members listed for 
possible lay-off, 

were all checked out. They [District officials] 
were looking at projector use. They figured that 
teachers who were showing a lot of films weren't 
teaching. They considered people's academic 
qualifications to teach subjects, their classroom 
control, anything that would justify cutting 
them. So all this [student survey] information — 
well, there's some specific feelings of mistrust. 

Later on, in the 1983-84 school year, contract negotiations 

between the local teachers' association and the District reached 

impasse. Teachers felt not only that the District's firm salary offer 
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was unsatisfactory, but it was extremely unreasonable. This situation 
aggravated the wounds opened by the reduction in force, leaving 
teachers feeling beleagured and unsupported. Problems that might 
otherwise have been interpreted by teachers as petty, bureaucratic 
Inefficiencies came to be viewed as evidence of the District's 
disregard for their professional status and needs. 

Whether all the comments quoted above reflected this state of 
affairs 1s problematic. It 1s probably reasonable to assume that some 
did, as the working-group teachers maintained. Furthermore, other 
faculty members' comments 1n the May 23 meetings made these links 
explicit. 



NOTE #7 (May 23 ,1984). The Period 2 small -group session 1s under 
way! Ken Sirotmk I UCLA) has explained the project's aims and 
elicited reactions. Many are negative. 

Soc ial Studies Teacher: "Part of the reaction you'll be seeing all 
city lies in the fact that we're 1n the classroom, we need supplies, 
materials, support of this kind and we're not getting 1t. 
Administrators are going off to meetings, 1ntellectual1zing about new 
educational ideas, but we can't get what we need to do our jobs. 11 
English Teacher: "I have a college prep class, British Literature, 
mere are dooks we need to read, which I ordered ages ago. They 
haven't arrived on time. How am I supposed to teach literature 
without books?" 

Social Studies Teacher: "The anthro. books I ordered 1n September for 
this semester haven't arrived yet." 

Second Social Studies Teacher: The same thing has happened with my 
global geography text. There are the problems we face, the practical 
day-to-day things. What am I going to do with more information? 
English Teacher: You're dealing with a very embittered staff. 
There's conflict between the teachers and the Hill [the District 
office]. Teachers here have been mistreated." 
Later, in tne sixth period session, a teacher echoed these themes. 
"Why should we get excited about Information systems, "he asked 
rhetorically, "when the District can't even order me my books. This 
[student survey] will just tax an already over- taxed system." 

By rough count, 68 Site A faculty and professional staff members 

attended the six small group sessions held on May 23 to introduce the 

systemic evaluation project and its student survey. Thirty three 
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(about 48%) took the floor to address their group "and UCLA partici- 
pants in particular. And of those 33 who spoke, 19 (roughly 57% of 
the speakers and 28% of the staff in attendence) articulated one or 
wore negative comments of the types quoted above. Nine others asked 
questions that can be construed as neutral in tone, e.g., "Are there 
standard procedures for administering [the student survey]? 11 ; "What 
should we do if the student doesn't know his CMS [Career Magnet 
School]?"; "When will you have the results back?" The remaining five 
speakers (Including two members of Site A - UCLA working group) 
offered comments which can be Interpreted as positive toward or 
supportive of the project 1n general or the student survey 1n 
particular, e.g., "Some of these questions look Interesting"; "I can 
vouch for Ken [Sirotnlk of UCLA]; I've worked with him before. He'll 
follow through." 

These simple counts should help to put the discussion 1n this 
section -- - and the demeanor of those at Site A toward the project 
into clearer perspective. Initial responses toward the project were 
universally enthusiastic. Faculty attitudes toward systemic 
evaluation activities are, apparently, mixed. Previous experiences 
have generated skepticism about the value of data collection, as well 
as resentment toward yet another activity that takes time away from 
the central business of teaching aad learning. The District adminis- 
tration and its creations (and this project can be interpreted as one 
by Site A staff) are viewed with antipathy and suspicion by many. All 
of this provides a challenge for the development of systemic evalua- 
tion at Site A. It is a challenge that can be met only in part by 
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presenting the faculty and staff with relevant information at relevant 
times in appropriate formats. Equally important, if not in fact more 
fundamental, is building an environment of greater trust and 
cooperation in a social system where mistrust and antagonism have 
recently been mere common. 

While this section has focused upon the initial attitudes of 
Site A faculty toward the systemic evaluation project and its reality 
test activities, it has shown some connections that appear to exist 
between those attitudes and the doings of the District 
administration. The next section examines the role of District 
leaders more explicitly and cites some reasons why their role is 
important in a school-based innovation. 

The Nature of District Support for the UCLA-Site A Project 

The last section began by suggesting that teachers 1 demeanor 
toward reality-test activities is one contextual feature that bears on 
the future of systemic evaluation at Site A High School. The nature 
of the District administration's support is another. In order tc 
understand the present status of that support and what it portends for 
the project, however, a bit of background is necessary. Thus, the 
discussion below opens with a brief review of the school and District 
commitment to the systemic evaluation reality test. Then, it moves on 
to consider how and why the District's coriani tment to the project is 
likely to impinge upon systemic evaluation's future at Site A. 
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The systemic evaluation reality test has had consistent, strong 
support from the Site A administration. The new principal (who 
assumed his position in September of 1983 and is now in his second 
year) appears to be enthusiastic about the project's potential ♦ He 
has taken the lead in securing release time for working-group teachers 
to participate in the project* He agreed to devote class time to the 
student survey and has scheduled the planned, three hour faculty 
meeting to examine the M At-A-Glanee M reporting forms and survey 
results* Despite a very busy schedule, he has attended all but one or 
two working-group meetings and several meetings among UCLA staff and 
personnel in the District office. Furthermore, he has expressed 
interest in the student survey findings, made suggestions for how they 
can be useful schoolwide, and cited them in support of several 
policies and decisions. The principal also has committed himself to 
involving the faculty a? a whole in examining andacting on schoolwide 
issues that project data help to make evident. All of this 
demonstrates his continuing support for the development of systemic 
evaluation at Site A. 

The assistant principal who championed the CASA system and acts 
as a member of the working-group is another key figure in the school 
administration's commitment to the project. A second assistant 
principal has become increasingly interested in systemic evaluation 
activities through recent months. 

In its commitment to project activities, the school 
administration has been generally reinforced by administrators at the 
District level. The Superintendent of the Valley Unified School 
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District involved it in a network of schools usually called The 
Partnership, sponsored by the UCLA Laboratory in School and Community 
Relations. It was through Partnership connections that the Valley 
District and Site A were identified as promising settings for the 
systemic evaluation reality test. The systemic evaluation concept 
appealed to the Superintendent, and he welcomed the project. 

The District's Assistant Superintendent for Instructional and 
Support Services is the Superintendent's designated contact for 
project issues. He has said on several occasions that building a 
comprehensive instructional information system "is something I've 
always wanted to do." With the Assistant Superintendent's approval, 
release time from the classroom for working-group teachers has been 
made possible, student surveys have been printed, survey data have 
been tabulated, and the District's information on students have been 
make available to UCLA staff. His approval of these activities 
testifies to District interest in and support of the systemic 
evaluation idea and reality-test activities. 

As the 1983-84 school year ended, the Assistant Superintendent 
affirmed the District's continuing "commitment 11 to the project. 
Ratifying that commitment, he promised approval of extra pay for 
teachers to join in a two-day working-group meeting in September of 
1984, just before the official opening of the new school' year. 

There is reason to believe that the Assistant Superintendent's 
interest in the systemic evaluation effort is more than nominal. 
During a meeting on September 27, 1984 to review project progress and 
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products (the !t At-A-Glance M forms and student survey data), he 

expressed enthusiasm with the forms* He became Involved 1n an 

animated discussion of survey findings. He also suggested other, 

school -level data that would be helpful to the District and to school 

counselors. When the head of District data processing suggested that 

some of this Information was already available, the Assistant 

Superintendent replied: 

I know you have 1t but the problem 1s to get 1t 
into a form like this , a form that's easy to read! 

He then went on enthusiastically to suggest various ways 1n which 

project designs and concepts could be adopted by the District for its 

information needs. 

The District's data processing personnel were equally affirmative 

in their reactions to the "At-A-Gl ance" forms and the information they 

contained. The director of the data processing unit, who often 

worries aloud about his need to keep Information and produce 

Information and produce reports "that nobody really wants," commented; 

The thing you [UCLA] guys did was go out and find 
what people wanted . That's what's Important... 
This is, this is a' pretty nice report! 

He added that the jun1or-h1gh-school counselors "would like this" and 

was soon immersed in a discussion of the techical prerequisites for 

producing the At-A-Glance reports on a routine basis. 

Thus, key District personnel have manifested genuine interest in 

the substance and products of the joint Site A - UCLA project. They 

have commented with some enthusiasm on their potential for wider use 

1n the District. Together with their 1 1m1ted-but-1mportant financial 



41 



Dorr-Bremme 40 



support (for survey duplication, data processing time, and 
substitutes to release working-group teachers from their classroom), 
this interest promises on-going commitment to the reality test, if not 
to systemic evaluation itself. 

Other events, however, blur the portrait of District support 
painted so far. First, the Assistant Superintendent equivocated on 
his commitment of extra pay for working-group teachers to attend the 
planned, early-September meetings. According to the Site A principal, 
he suggested in early August that the funds for this extra pay could 
come from a special state grant the school was hoping to win, but 
added that "we'll find some way" to compensate them for their 
additional work time. Later in August, a phone call to the principal 
revealed that the Associate Superintendent had not yet formally agreed 
to such compensation. The principal delayed inviting teachers to the 
two-day September meeting pending a District commitment to compen- 
sation. Plans to expand the number of teachers in the working group 
were also delayed. In yet another phone call to the principal three 
days before the scheduled meeting in September, he reported, "I don't 
know about compensation. I think we have some money here in the 
school that will cover [the teachers 1 ] lunch." Ultimately, the meet- 
ing took place, but for four tnurs instead of the scheduled two days. 
The working-group was not enlarged; only the administrators and four 
of the five faculty members who had been working with the group all 
along attended. Given the ambiguity surrounding the compensation 
issues, it seemed inappropriate to increase the group's size or to ask 
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teachers to donate more of their time in the days just before school 
opened. The state grant application (mentioned above) was not 
successful. No District funds were forthcoming. 

Second, during the September 27, 1984 meeting cited earlier, the 
Assistant Superintendent maintained that "there's no extra money" for 
various District office work in support of systemic evaluation. UCLA 
staff and the Site A principal had explained their interest in 
developing and administering a teacher survey, using District data 
programmers to produce "Students-At-A-Glance " (and possibly "Class- 
At-A-Glance") forms for all Site A teachers' second semester classes, 
and continuing working-group meetings with ten faculty members rather 
than the present five. The Assistant Superintendent was most 
concerned, in the absence of "extra money," about the costs of release 
time for teachers and the level of effort District data programmers 
would need to invest. "My feeling is," he said, "we've invested time 
in this and we should go the next step, but we can't omit the fact 
that we don't have a lot of bucks." In the end, the Associate 
Superintendent requested the director of data processing to give him 
"an ideal of how much time this will take. If it's two months of two 
programmers' time, maybe we can't do it." He also directed the Site A 
principal "to come up with an estimate of the release time" for 
teachers that the project would require through the 1984-85 school 
year. "I'd like to go to the Cabinet with the whole package, but I 
need to tell them what it would cost." 

These time-cost estimates were subsequently provided, but by 
mid-October no response from the Assistant Superintendent on any of 
these matters had been received at Site A or UCLA. 
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Third, the ambiguous nature of District support for the project 
demonstrated by these events may be part of a larger pattern. In the 
veiw of several persons with whom UCLA project participants have 
spoken, the District administration has a tendency to let schools do 
whatever they wish, but to drop projects once the extra District 
monies or outside funds for those school -initiated projects are gone* 

In overview, then, District support continued to be quite solid 
through the 1983-84 school year. As the 1984-85 year opened and UCLA 
staff made a tactical decision to encourage the District to assume 
greater responsibility for systemic evaluation, however, the nature 
and extent of that support became less clear. 

The foregoing account should demonstrate that a districts 1 sup- 
port for a school-level innovation is subject to on-going negotia- 
tion. As circumstences at the district level evolve, priorities 
change; new demands on district resources arise. Earlier "commit- 
ments" to particular projects need to be re-examined by district 
administrators in light of changing circumstances. At the same time, 
school personnel cannot always specify in advance exactly what a 
development project will require in terms of district support. UCLA 
participants were careful to detail what, in general, the project 
would require of the Valley Unified School District and Site A. 
Release time for teachers throughout the reality test was mentioned. 
So, too, were data processing time and clerical support (for duplicat- 
ing surveys, etc.). But until longer-term plans and particular data 
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collection and reporting procedures emerged from the developmental 
process, no one could say with precision just how many teachers would 
need to be released from class assignments for how many hours, how 
many pages would need to be duplicated by what deadlines, or how many 
District programmers would need to work for how long in support of the 
project. Thus, just as changing District political and economic 
circumstances open the door to renegotiation of levels of support for 
a given innovation, so too does the evolution of needs at the school 
level in any formative or developmental project. There is nothing 
especially unusual, then, in the sequence of events described above. 

Considered in broader perspective, the nature of District support 
is likely to be a key contextual factor in the long-term maintenance 
of systemic evaluation at Site A.* The support and collaboration of 
both school and district leadership tends to be critical* in the 
maintenance of innovative educational programs {e.g., Berman & 
McLaughlin, 1977). This holds true when the innovation in question is 
an instructional information system. Bank and Williams (1981, 1983), 
for instance, have studied a small number of school districts that 
have made unusual advances in linking testing and evaluation data with 
instructional planning and decision making. In none of these cases 

*In the short run, UCLA resources through the Center for the Study 
of Evaluation from the National Institute of Education and from the 
Laboratory in School and Community Relations will sustain the reality 
test. Of course, the project's development activities will probably 
need to be -adjusted if the District provides no financial support. 
There is, however, every likelihood that Valley will continue to bear 
the costs of at least some reality-test activities. 
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was the mere presence of relevant and readily utilizable information 
sufficient in itself to sustain the links or guarantee the 
information's use. In every district studied, there were one or two 
idea champions at the district level who took the lead in sustaining 
the linkage system. Each district devoted considerable resources to 
structures that supported the system. Most maintained on-going staff 
development which helped teachers learn how to interpret and act on 
the available information in their everyday activities. All created 
(or capitalized upon extant) organizational arrangements within which 
school personnel were empowered to use the information in making 
choices among alternative educational policies and practices. In 
short, these exemplary districts reified their commitment to and 
support of instructional information systems by institutionalizing 
them in a network of mutually interdependent and mutually sustaining 
activities carried out collaboratively in both school and district 
settings. 

Now, the systemic evaluation effort differs from those studied by 
Bank and Williams. The latter were district generated and 
districtwide in scope; implementation moved "top down," from district 
office to the schools. In contrast, the joint UCLA-Site A project 
tests a school -basf.d, "ground-up" approach to the development of 
information systems. At the school level, some of the functions Bank 
and Williams identified as components of successful instructional 
information systems have begun to emerge. While UCLA participants 
were the original idea champions, the Site A principal, an assistant 
principal, and the cadre of working-group teachers have begun to 
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assume that function* (They have enthusiastically taken 
responsibility for the upcoming faculty meeting to present the project 
and Its products to the entire staff, for Instance.) Through 
working-group sessions, they have begun to learn how to think about 
information and Its uses 1n Instructional planning and decision 
making* 

Nevertheless, it seems likely that the Valley Unified School 
District will need to take at least some steps along the path marked 
out by Bank and Williams' findings if Site A is to maintain whatever 
information system UCLA helps the school develop. As noted in the 
last section, teachers at Site A are skeptical about the value of 
Information. They define their relationship with the District 
administration as one of conflict, rather than collaboration. Thus, 
it seems unlikely that Site A faculty will give their time and energy 
to systemic evaluation without guarantees of District support. 
Furthermore, as the role of UCLA personnel gradually changes from one 
of initiation and leadership to one of support and study, the 
District's commitment will become more critical. Even though Site A 
working-group members may assume leadership, they will need help and 
guidance from the District office. They will need, at a minimum, the 
resources required for data collection, processing, and reporting in 
relevant formats. They will also need time to consider and discuss 
school -level data; to act on 1t in ways they deem appropriate for 
school renewal; and to review, revise and (thus) maintain their 
Information system. They will also need problem-solving advice on 
technical and social organizational issues. All of this, 1t now 
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seems, will have to come directly or indirectly from the District 
office. Given the District administration's history and present 
ambiguous demeanor, it remains problematic how much of this support 
the District will choose to provide, 

CONCLUSION 

This paper has discussed four key social contextual issues that 
have emerged during the early phases of the joint UCLA-Site A systemic 
evaluation project: 

(1) The current status of the Valley-Site A 
information system (CASA); 

(2) Site A High teachers 1 clinical perspective on 
information and features of the school's social, 
organization; 

(3) Faculty-district relations as they impinge on the 
development of systemic evaluation at Site A; and 

(4) The nature of District support for the UCLA-Site A 
"reality test 11 and what it portends for the 
future. 

In documenting and describing each of these issues, the paper has 
been a progress report, not an account of case study findings. 
Nevertheless, it has suggested at appropriate points that the 
phenomena observed during the reality test thus far are at least 
similar to those found in other schools and school districts. The 
"under-utilization" of extant information is not unique to Site A, nor 
are the circumstances which appear to have led to the restricted use 
of CASA. Clinical thinking about information is sustained by the 
atomistic nature of Site A's organizational structure, but many 
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comprehensive high schools share this structure and a clinical 
perspective toward student information 1s widespread. Bad experiences 
with information — experiences 1n which the t1me-and-effort costs far 
exceed discernible benefits — are common 1n schools. So too are 
instances of Di strict- faculty conflict and mistrust generated by 
contract negotiations and reductions 1n staff. Policy changes and 
vacillations in District support for schools 1 projects are usual, not 
extraordinary. Any "ground up" school-based Information system, then, 
1s likely to encounter such Issues 1n the course of Its development. 
As this paper has Illustrated, such systems do not succeed or fall by 
virtue of their independent merit: on the basis of their quality of 
convenience or relevance alone. Rather, an Information system and Its 
social context are interdependent 1n dynamic, ecological balance. The 
social organizational arrangements of the school and district shape 
and sustain (or fall to sustain) the Information system; and it, in 
turn, helps to shape and sustain (or fall to sustain) the arrangements 
of the school and district. It is the nature and features of this 
process which the systemic evaluation project hopes to Identify and 
learn from as the reality test proceeds. 
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